System and method for photographing moving subject by means of camera, and acquiring actual movement trajectory of subject based on photographed image

ABSTRACT

The present invention relates to a system and method for photographing a moving subject by means of a camera, and acquiring the actual movement trajectory of the subject on the basis of the photographed image. One embodiment of the present invention provides a method for acquiring the movement trajectory of a subject, the method comprising: a step for photographing a moving subject by means of a camera; a step for collecting, from the camera, information on multiple images of the subject and the positions of the images on a camera image frame, and collecting information on the size of the images; and a step for acquiring the movement trajectory of the subject on the basis of the information collected.

CROSS-REFERENCE TO RELATED APPLICATION

Cross-reference is made to another application entitled, “SYSTEM ANDMETHOD FOR PHOTOGRAPHING MOVING SUBJECT BY MEANS OF FIXED CAMERA, ANDACQUIRING PROJECTION IMAGE OF ACTUAL MOVEMENT TRAJECTORY OF SUBJECTBASED ON PHOTOGRAPHED IMAGE” and filed on the same date as thisapplication by the present applicant. Therefore, those skilled in theart to which the present invention pertains may refer to the inventionis of the above other application to understand or practice the presentinvention, or vice versa.

FIELD OF THE INVENTION

The present invention relates to a system and method for photographing amoving subject by means of a camera and acquiring an actual movementtrajectory of the subject on the basis of the photographed imagesthereof.

BACKGROUND

Virtual golf systems are widely spreading which allow golfers tovirtually play golf at low cost in downtown areas and the like. Thebasic concept of such virtual golf systems is to acquire a number ofphotographed images of a golf ball upon being hit by a golfer, measurephysical quantities of the golf ball on the basis of the trajectory,interval, size and the like thereof, perform a simulation of the shot,and display a result of the simulation on a screen.

Conventionally, in order to recognize information on the above physicalquantities, particularly the moving direction or moving speed of thegolf ball, techniques using multiple cameras synchronized tosimultaneously photograph the golf ball have been largely employed.

However, the techniques using such cameras have disadvantages ofincreasing system production cost or restricting the arrangement of thecameras.

Therefore, following the introduction of a remarkable virtual golfsystem in Korean Patent No. 1048864 (entitled, “METHOD OF MEASURINGPHYSICAL QUANTITIES OF OBJECT BY USING SINGLE LIGHT SOURCE AND PLANARSENSOR UNIT AND VIRTUAL GOLF SYSTEM UTILIZING SAME”) (the contents ofwhich are incorporated herein by reference in its entirety), theinventor(s) now present a novel technique to combine with such virtualgolf systems, other virtual golf systems, systems for virtually playingother kinds of sports (e.g., baseball, football, etc.), virtual realitysystems for subjects other than balls, or the like to enable them toproduce sufficiently good simulation results just using some economicalcomponents.

SUMMARY OF THE INVENTION

One object of the present invention is to acquire an actual movementtrajectory of a moving ball on the basis of photographed images of theball.

Another object of the invention is to enable a virtual sport system toproduce a sufficiently good simulation result just using some economicalcomponents.

Yet another object of the invention is to acquire an actual movementtrajectory of a moving subject on the basis of photographed images ofthe subject.

According to one aspect of the invention to achieve the objects asdescribed above, there is provided a method for acquiring a movementtrajectory of a subject, comprising: photographing a moving subject bymeans of a camera; collecting from the camera a plurality of images ofthe subject and information on their locations on a camera image frame,and collecting information on the size of the plurality of images; andacquiring a movement trajectory of the subject on the basis of thecollected information.

In addition, there may be provided other methods and systems toimplement the present invention.

According to the invention, an actual movement trajectory of a movingball may be acquired on the basis of photographed images of the ball.

According to the invention, a virtual sport system may produce asufficiently good simulation result just using some economicalcomponents.

According to the invention, an actual movement trajectory of a movingsubject may be acquired on the basis of photographed images of thesubject.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of the overall configuration of a virtualgolf system according to one embodiment of the invention.

FIG. 2 is a detailed diagram of the internal configuration of aphotographing unit 100 according to one embodiment of the invention.

FIG. 3 is a detailed diagram of the internal configuration of asimulator 200 according to one embodiment of the invention.

FIG. 4 shows one concept employed in the present invention.

FIG. 5 illustrates a line of intersection obtained when using twocameras according to one embodiment of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

In the following detailed description of the invention, references aremade to the accompanying drawings that show, by way of illustration,specific embodiments in which the invention may be practiced. Theseembodiments are described in sufficient detail to enable those skilledin the art to practice the invention. It is to be understood that thevarious embodiments of the invention, although different from eachother, are not necessarily mutually exclusive. For example, specificshapes, structures, or characteristics described herein may beimplemented as modified from one embodiment to another embodimentwithout departing from the spirit and the scope of the invention.Furthermore, it shall be understood that the locations or arrangementsof individual elements within each embodiment may be also modifiedwithout departing from the spirit and the scope of the invention.Therefore, the following detailed description is not to be taken in alimiting sense, and the scope of the invention is to be taken asencompassing the scope of the appended claims and all equivalentsthereof. In the drawings, like reference numerals refer to the same orsimilar elements throughout the several views.

Hereinafter, various preferred embodiments of the present invention willbe described in detail with reference to the accompanying drawings toenable those skilled in the art to easily implement the invention.

Configuration of Overall System

FIG. 1 is a schematic diagram of the overall configuration of a virtualgolf system according to one embodiment of the invention.

As shown in FIG. 1, the virtual golf system may be configured tocomprise a shot unit 10, a photographing unit 100, a simulator 200 and adisplay device 300.

First, the shot unit 10 according to one embodiment of the invention maybe a part on which a golfer steps up to place and hit a golf ball whenusing the virtual golf system. The shot unit 10 may comprise a knownswing plate, the tilt angle of which may be adjusted. It will be notethat when the invention is applied to other kinds of virtual sportsystems, those skilled in the art may modify the configuration of theshot unit 10 and, if necessary, those of other components associatedtherewith to suit to the characteristics of the corresponding sports.

Next, the photographing unit 100 according to one embodiment of theinvention may comprise a plurality of cameras (preferably infraredcameras) so that each of them may perform a function to acquire a numberof images of the moving golf ball.

As shown in FIG. 1, the photographing unit 100 may be disposed in alocation to look down at the moving golf ball, while it may be disposedin other locations. Preferably, the photographing unit 100 is fixed tothe structure as shown in FIG. 1 so that the location and view directionthereof may be invariable. Meanwhile, the plurality of cameras in thephotographing unit 100 are preferably spaced apart from each other.

The detailed configuration of the photographing unit 100 will be furtherdescribed later with reference to FIG. 2.

Next, the simulator 200 according to one embodiment of the invention mayperform a function to receive from the photographing unit 100 aplurality of images of the moving golf ball acquired by each of theplurality of cameras, acquire an actual movement trajectory of the golfball, and perform a simulation on the basis thereof.

The simulator 200 may communicate with the photographing unit 100 andthe display device 300, and may comprise a dedicated processor forvirtual golf simulation. The dedicated processor may be provided withmemory means and have numerical operation and graphics processingcapabilities. The simulator 200 may be similar to conventional virtualgolf simulation devices.

The configuration of the simulator 200 will be further described laterwith reference to FIG. 3.

Lastly, the display device 300 according to one embodiment of theinvention may perform a function to display a result of the numericaloperation or graphics processing of the simulator 200. The displaydevice 300 may display images via display means, and may preferably beconfigured with a screen, which absorbs the impact of the hit golf balland does not emit light directly, and a projector to output images onthe screen.

Configuration of Photographing Unit

Hereinafter, the internal configuration of the photographing unit 100according to one embodiment of the invention and the functions of therespective components thereof will be described.

FIG. 2 is a detailed diagram of the internal configuration of thephotographing unit 100 according to one embodiment of the invention.

As shown in FIG. 2, the photographing unit 100 may be configured tocomprise a camera unit 110, a communication unit 120 and a control unit130.

According to one embodiment of the invention, at least some of thecamera unit 110, the communication unit 120 and the control unit 130 maybe program modules to communicate with the simulator 200. The programmodules may be included in the photographing unit 100 in the form ofoperating systems, application program modules or other program modules,while they may be physically stored in a variety of commonly knownstorage devices. Further, the program modules may also be stored in aremote storage device that may communicate with the photographing unit100. Meanwhile, such program modules may include, but not limited to,routines, subroutines, programs, objects, components, data structuresand the like for performing specific tasks or executing specificabstract data types as will be described below in accordance with thepresent invention.

First, the camera unit 110 may comprise two or more cameras that mayoptically acquire images of the moving golf ball. Each of the pluralityof cameras may acquire a plurality of images of the moving golf ballfrom various viewpoints. In this case, there is no need to synchronizethe plurality of cameras.

The camera unit 100 may transmit to the control unit 130 each of thephotographed images of the moving golf ball together with identificationinformation on the corresponding camera and information on itscoordinate on the corresponding camera image frame. This processing maybe performed with respect to the plurality of cameras and thecorresponding photographed images. In connection with the coordinate onthe camera image frame, reference may be made to the aforementionedcross-referenced application.

The cameras of the camera unit 110 may be fixed to the structure asdescribed above and then calibrated as necessary. The above calibrationmay be directed to calibrating the distortion of a camera image sensoror a lens according to conventional techniques.

Next, the communication unit 120 may perform a function to mediate datatransmission/receipt between the control unit 130 and the simulator 200.Although there is no particular limitation on the communication modalitythat may be employed by the communication unit 120, wired communicationsuch as wired LAN communication and cable communication, or wirelesscommunication such as wireless LAN communication, infraredcommunication, RF communication and Bluetooth communication maypreferably be employed.

Lastly, the control unit 130 may transmit to the simulator 200 variousinformation on the photographed images of the moving golf ball receivedfrom the camera unit 110.

Meanwhile, according to the unique principle of the present invention tobe described below, an actual movement trajectory of the golf ball maybe acquired even when the plurality of cameras are not synchronized tosimultaneously photograph the golf ball. Therefore, when all N camerasof the camera unit 110 have a photographing period of T, the controlunit 130 may control each of the cameras to photograph in sequence at atime interval of T/N as if cameras having a shorter photographing periodof T/N (i.e., having a higher performance in terms of temporalresolution) were used.

Further, when all the N cameras of the camera unit 110 have aphotographing period of T, the control unit 130 may control at leastsome of the cameras to photograph at predetermined time intervals as ifcameras having a photographing period shorter than T (i.e., having ahigher performance in terms of temporal resolution) were used.

Meanwhile, even when at least some of the photographing periods of the Ncameras of the camera unit 110 differ from each other, the control unit130 may control at least some of the cameras to photograph atpredetermined time intervals, so that more photographing data may beobtained.

Configuration of Simulator

Hereinafter, the internal configuration of the simulator 200 accordingto one embodiment of the invention and the functions of the respectivecomponents thereof will be described.

FIG. 3 is a detailed diagram of the internal configuration of thesimulator 200 according to one embodiment of the invention.

As shown in FIG. 3, the simulator 200 according to one embodiment of theinvention may be configured to comprise a simulation unit 210, a datastorage unit 220, a communication unit 230 and a control unit 240.

According to one embodiment of the invention, at least some of thesimulation unit 210, the data storage unit 220, the communication unit230 and the control unit 240 may be program modules to communicate withthe photographing unit 100 or the display device 300. The programmodules may be included in the simulator 200 in the form of operatingsystems, application program modules or other program modules, whilethey may be physically stored in a variety of commonly known storagedevices. Further, the program modules may also be stored in a remotestorage device that may communicate with the simulator 200. Meanwhile,such program modules may include, but not limited to, routines,subroutines, programs, objects, components, data structures and the likefor performing specific tasks or executing specific abstract data typesas will be described below in accordance with the present invention.

First, the simulation unit 210 may receive from the photographing unit100 various information on the photographed images of the moving golfball as described above. Using the information, the simulation unit 210may acquire an actual movement trajectory and moving speed of the golfball through the following operational processing. Reference will bemade to FIG. 4. (FIG. 4 shows one concept employed in the presentinvention.)

1. Acquiring a Plane Determined by the Location Point of a Camera andthe Points at which the Location Points of the Golf Ball When BeingPhotographed by the Corresponding Camera are Projected Onto a BackgroundRegion

In FIG. 4, Pc(i) represents the location point of an i^(th) camera ofthe plurality of cameras. The simulation unit 210 may already be awareof the coordinate of Pc(i).

Further, Pb(i, n) to Pb(i, n+2) represent the location points at whichthe golf ball is actually located when being photographed by the i^(th)camera for the n^(th) to n+2^(th) times. They are on a straight line Lt,which is an actual movement trajectory of the golf ball. (Here, althoughthe actual movement trajectory of the golf ball is not necessarily astraight line, it should be noted that the line Lt may be assumed to bestraight because a trajectory of a golf ball appears to be almoststraight immediately after being hit.) Further, the straight line Lt mayrepresent an approximate movement trajectory of the golf ball for agiven time span.

Pr(i, n) represents the point at which Pb(i, n) is projected from aviewpoint of the i^(th) camera onto a ground surface as a backgroundregion. According to the principle of the invention disclosed in thecross-referenced application, the simulation unit 210 may determine thecoordinate on the ground surface of the above point based on, forexample, the coordinate on the camera image frame of the image in whichthe golf ball located at Pb(i, n) is photographed by the i^(th) camera,and a coordinate conversion matrix which may already be obtained for thei^(th) camera. In the same manner, Pr(i, n+1) and Pr(i, n+2) representthe points at which Pb(i, n+1) and Pb(i, n+2) are projected from theviewpoint of the i^(th) camera onto the ground surface. The coordinateson the ground surface of those points may also be determined.

Therefore, the simulation unit 210 may acquire a plane uniquelydetermined by Pc(i) and a straight line Lb(i) formed by Pr(i, n) toPr(i, n+2).

2. Acquiring a Plurality of Planes Containing a Movement Trajectory ofthe Golf Ball

Those planes may contain the straight line Lt. Further, the simulationunit 210 may also perform the first processing with respect to ani+1^(th) camera. Naturally, the simulation unit 210 may also perform thefirst processing with respect to an i+2^(th) camera, as necessary. Thus,the simulation unit 210 may acquire one more plane containing thestraight line Lt at each time of performing the processing.

3. Acquiring a Movement Trajectory of the Golf Ball

The simulation unit 210 may acquire a line of intersection (shown by thethick line in FIG. 5) of the planes acquired from the second processing.(FIG. 5 illustrates a line of intersection obtained when using twocameras according to one embodiment of the invention.) This linecoincides with the straight line Lt, which is the actual movementtrajectory of the golf ball.

4. Acquiring a Moving Speed of the Golf Ball

The simulation unit 210 may also obtain the coordinate of Pb(i, n) usingthe straight line Lt acquired from the third processing and the knowncoordinates of Pc(i) and Pr(i, n). In the same manner, the coordinate ofPb(i, n+1) may also be obtained. Thus, the simulation unit 210 mayacquire an actual moving speed of the golf ball by dividing the distancebetween the obtained two coordinates by the photographing period of thei^(th) camera.

Meanwhile, although it has been illustrated above that the simulationunit 210 may operationally acquire the movement trajectory and movingspeed of the golf ball, it will be apparent to those skilled in the artthat at least a part of the operational procedure may be performed byother operational components such as the control unit 130 of thephotographing unit 100.

Next, the data storage unit 220 may store various information used orderived in the above-described operational procedure, or otherinformation required for the simulation. The data storage unit 220 maycomprise a computer-readable recording medium.

Next, the communication unit 230 may perform a function to enable datatransmission/receipt to/from the simulation unit 210 and the datastorage unit 220. Although there is no particular limitation on thecommunication modality that may be employed by the communication unit230, wired communication such as wired LAN communication and cablecommunication, or wireless communication such as wireless LANcommunication, infrared communication, RF communication and Bluetoothcommunication may preferably be employed.

Lastly, the control unit 240 may perform a function to control data flowamong the simulation unit 210, the data storage unit 220 and thecommunication unit 230. That is, the control unit 240 according to thepresent invention may control data flow into/out of the simulator 200 ordata flow among the respective components of the simulator 200, suchthat the simulation unit 210, the data storage unit 220 and thecommunication unit 230 may carry out their particular functions,respectively.

Applications

By applying the principle of the present invention, an actual movementtrajectory of a golf ball may be acquired even when a plurality ofimages photographed by one of a plurality of cameras and informationassociated therewith are only available (e.g., when only one camera hasacquired valid images due to the bounce of the golf ball) or when onlyone camera is employed. Hereinafter, the above application of theinvention will be discussed.

The simulation unit 210 of the simulator 200 may further employinformation on the size of the images of the golf ball received from thecontrol unit 130 of the photographing unit 100. The information may bethe numbers of pixels occupied by the images of the golf ball within acorresponding camera image frame, or the lengths of diameters that theimages of the golf ball exhibit in a coordinate plane on thecorresponding camera image frame. In addition, other information relatedto the size of the images of the golf ball may also be employed.

When images photographed by a plurality of cameras and informationassociated therewith are available, the simulation unit 210 may alsoextract information on the size of the images of the golf ball and putthe correlation between the size of each of the images and the actualmovement trajectory into a database in the data storage unit 220, inadvance. The information on the correlation in the database may beupdated in real time while the virtual golf system is used by a golferor else during a system test.

Therefore, when a plurality of images photographed by one camera andinformation associated therewith are only available, the simulation unit210 may first acquire a plane related to the camera as described withthe above embodiments, and then acquire an actual movement trajectory(i.e., a trajectory assumed to be a straight line) of the golf ball withreference to the above database. Further, the simulation unit 210 mayalso acquire an actual moving speed of the golf ball on the basis of theobtained plane and movement trajectory, as described with the aboveembodiments.

Meanwhile, after once acquiring the actual movement trajectory of thegolf ball according to the above embodiments when images photographed bya plurality of cameras and information associated therewith areavailable, the simulation unit 210 may verify the accuracy of theacquired movement trajectory by employing the information on the size ofeach of the images.

Although it has been mainly described above that the system of thepresent invention is a virtual golf system, it will be apparent to thoseskilled in the art that the technical principle and configuration of theinvention may be applied to all kinds of virtual reality systems (e.g.,virtual baseball systems or virtual football systems) requiringsimulation of the movement of a subject.

The embodiments according to the present invention as described abovemay be implemented in the form of program instructions that can beexecuted by various computer components, and may be stored on acomputer-readable recording medium. The computer-readable recordingmedium may include program instructions, data files, data structures andthe like, separately or in combination. The program instructions storedon the computer-readable recording medium may be specially designed andconfigured for the present invention, or may also be known and availableto those skilled in the computer software field. Examples of thecomputer-readable recording medium include the following: magnetic mediasuch as hard disks, floppy disks and magnetic tapes; optical media suchas compact disk-read only memory (CD-ROM) and digital versatile disks(DVDs); magneto-optical media such as floptical disks; and hardwaredevices such as read-only memory (ROM), random access memory (RAM) andflash memory, which are specially configured to store and executeprogram instructions. Examples of the program instructions include notonly machine language codes created by a compiler or the like, but alsohigh-level language codes that can be executed by a computer using aninterpreter or the like. The above hardware devices may be changed toone or more software modules to perform the operations of the presentinvention, and vice versa.

Although the present invention has been described in terms of specificitems such as detailed elements as well as the limited embodiments andthe drawings, they are only provided to help general understanding ofthe invention, and the present invention is not limited to the aboveembodiments. It will be appreciated by a person of ordinary skill in theart that various modifications and changes may be made from the abovedescription.

Therefore, the spirit of the present invention shall not be limited tothe above-described embodiments, and the entire scope of the appendedclaims and their equivalents will fall within the scope and spirit ofthe invention.

What is claimed is:
 1. A computer-implemented method for acquiring amovement trajectory of a subject, said method comprising using aprocessor to perform steps of: photographing a moving subject by meansof a camera; collecting a plurality of images of the subject andinformation on their locations on a camera image frame, and collectinginformation on the sizes of the plurality of images; and acquiring amovement trajectory of the subject on the basis of the collectedinformation on the locations and the sizes of the plurality of images,wherein the step of acquiring a movement trajectory of the subjectcomprises uniquely determining one plane related to the movementtrajectory of the subject on the basis of the information on thelocations of the plurality of images, wherein the step of acquiring amovement trajectory of the subject is performed on the basis of the oneplane with reference to a database in which correlations between thesizes of the plurality of images and the movement trajectory of thesubject are electronically pre-stored, and wherein the information onthe sizes of the plurality of images includes information on the lengthsof diameters that the plurality of images of the subject exhibit in acoordinate plane on the camera image frame.
 2. A method as claimed inclaim 1, wherein the information on the sizes of the plurality of imagesfurther includes information on the numbers of pixels occupied by theplurality of images within the camera image frame.
 3. A system foracquiring a movement trajectory of a subject, comprising: a camera tophotograph a moving subject; a simulator to collect from the camera aplurality of images of the subject and information on their locations ona camera image frame, collect information on the sizes of the pluralityof images, and acquire a movement trajectory of the subject on the basisof the collected information on the locations and the sizes of theplurality of images; and a database to electronically pre-storecorrelations between the sizes of the plurality of images and themovement trajectory of the subject, wherein the simulator uniquelydetermines one plane related to the movement trajectory of the subjecton the basis of the information on the locations of the plurality ofimages, and wherein the simulator acquires the movement trajectory ofthe subject on the basis of the one plane with reference to thedatabase, and wherein the information on the sizes of the plurality ofimages includes information on the lengths of diameters that theplurality of images of the subject exhibit in a coordinate plane on thecamera image frame.